1. Technical Field
The present invention relates to video processing and more particularly to systems and methods for robust and efficient foreground analysis of video data.
2. Description of the Related Art
Robust detection of moving objects in video streams is a significant issue for video surveillance. Background subtraction (BGS) is a conventional and effective approach to detect moving objects in the stationary background. To detect moving objects in a dynamic scene, adaptive background subtraction techniques have been developed. See C. Stauffer and W. E. L. Grimson, “Adaptive Background mixture Models for Real-time Tracking”, CVPR99, June, 1999. Stauffer et al. modeled each pixel as a mixture of Gaussians and used an on-line approximation to update the model. Their system can deal with lighting changes, slow-moving objects, and introducing or removing objects from the scene.
Monnet et al. in A. Monnet, A. Mittal, N. Paragios, and V. Ramesh, “Background Modeling and Subtraction of Dynamic Scenes”, In Proc. of International Conference on Computer Vision (ICCV), 2003, Pages 1305-1312, proposed a prediction-based online method for the modeling of dynamic scenes. Their approach has been tested on a coastline with ocean waves and a scene with swaying trees. However, they need hundreds of images without moving objects to learn the background model, and the moving object cannot be detected if they move in the same direction as the ocean waves.
Mittal and Paragios, in A. Mittal and N. Paragios, “Motion-based Background Subtraction using Adaptive Kernel Density Estimation,” Proceedings on Computer Vision and Pattern Recognition (CVPR04), 2004, presented a motion-based background subtraction by using adaptive kernel density estimation. In their method, optical flow is computed and utilized as a feature in a higher dimensional space. They successfully handled the complex background, but the computation cost is relatively high.
More recently, L. Li, W. Huang, I. Y. H. Gu, and Q. Tian, “Statistical Modeling of Complex Backgrounds for Foreground Object Detection”, IEEE Transaction on Image Processing, Vol. 13, No. 11, 2004, proposed a Bayesian framework that incorporates spectral, spatial, and temporal features to characterize the background appearance at each pixel. Their method can handle both the static and dynamic backgrounds, and good performance was obtained on image sequences containing targets of interest in a variety of environments, e.g., offices, public buildings, subway stations, campuses, parking lots, airports, and sidewalks.
Although many researchers focus on the background subtractions, few papers can be found in the literature for foreground analysis. R. Cucchiara, C. Grana, M. Piccardi, and A. Prati, “Detecting Moving Objects, Ghosts, and Shadows in Video Streams,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 25, No. 10, October 2003, analyzed the foreground as moving object, shadow, and ghost by combining the motion information. The computation cost is relatively expensive for real-time video surveillance systems because of the computation of optical flow.
Recently, the mixture of Gaussians method is becoming popular because it can deal with slow lighting changes, periodical motions from clutter background, slow moving objects, long term scene changes, and camera noises. But it cannot adapt to the quick lighting changes and cannot handle shadows well. A number of techniques have been developed to improve the performance of the mixture of Gaussians method. See, e.g., H. Eng, J. Wang, A. Kam, and W. Yau, “Novel Region-based Modeling for Human Detection within High Dynamic Aquatic Environment,” Proceedings on Computer Vision and Pattern Recognition (CVPR04), 2004 and O. Javed, K. Shafique, and M. Shah, “A Hierarchical Approach to Robust Background Subtraction using Color and Gradient Information,” IEEE Workshop on Motion and Video Computing, 2002.